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An apparatus and method is disclosed for providing a user (106) 
with task-specific information that includes a portable instruction system 
that may be worn by a user (106), and includes, a computer (102) 
sufficiently lightweight and designed to be worn by a user (106) to which 
a memory has been connected. The system includes a display device 
(108) that can receive display signals from die computer (102) for visual 
display to the user (106) and an input device (122) by which the user 
(106) enters commands to the computer. An instructional program is 
provided that the computer (102) accesses and stores in a memory in 
response to a user (106) command and displays information concerning 
a task to be performed by the user (106) on the display device (108) in 
response to conrunands from the user. 
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SYSTEM, APPARATUS AND METHOD FOR PROVTOING A PORTABLE 
CUSTOMIZABLE MAINTENANCE SUPPORT INSTRUCTION SYSTEM 

TECHNICAL FIELD OF THE INVENTION 
The present invention relates in general to the field of real-time instruction 
and maintenance support, and more particularly, to the use of a portable system, 
5 apparatus and method for the remote repair or maintenance of complex mechanical 
systems that is hands-fi^e and which is customizable on-site for a particular piece of 
equipment, process, operation or system. 
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BACKGROUND OF THE INVENTION 

Without limiting the scope of the mvention, its background is described in 
connection with a portable computer for use during repair, support and maintenance 
support operations, as an example. 

In order to remain competitive, many industries, such as the aircraft industry, 
have increasingly automated their assembly processes through the use of 
computer-controlled equipment The increase in complexity of the mechanical and 
computer systems in the airline, automobile, and other industries that rely heavily on 
automation, have led to the need for an increasingly sophisticated labor force. 
Unfortunately, the labor markets are failing to meet the demand for highly competent 
repair and maintenance employees at reduced costs. As equipment becomes more 
complex, the diagnosis of faults and maintenance of the equipment have, likewise, 
become more complex. 

During the repair and maintenance process, operators are often responsible for 
diagnosing and resolving problems. Since the processing that occurs with a fiilly 
functional piece of equipment is stopped during the repair and maintenance time, it 
is important that the operator diagnose and resolve the fault expeditiously. The 
problem is fiirther enhanced by the need for specialized labor that often must be 
brought fi-om a remote site or another city to perform the repair procedure. 

In the case of preventative maintenance, a similar problem to that of repair 
work is the need to file reports detailing the procedures and repairs accompUshed. 
Part of the task of reporting repair and maintenance work is the need to specify the 
parts replaced, as well as, reordering of new parts to maintain in inventory. Typically, 
preventative maintenance is scheduled based upon manufacturer estimates for the 
lifetime of various components. In order to accurately and completely follow and 
maintain these schedules, the operator must not only be well-versed in both 
equipment part location and maintenance procedures, but in correctly filing-out and 
filing the reports in a timely manner. 

Furthermore, the reporting process must allow the user and others to retrieve, 
view, enhance, and interpret the information. One such procedure is to dictate or 
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"annotate" the diagnosis to produce a final report that may be accessed by other 
maintenance personnel. More modem systems, such as those in medical record 
keeping, involve the use of multiple media data with static graphic drawings on the 
report. These prior art techniques, however, use visual annotations to annotate 
5 process and procedures along with the annotator^s voice, and to playback the recorded 
result. Then, the annotations must be transcribed manually in order to produce a 
complete report. 

A system for repair is disclosed in United States Letter Patent No. 5,754,451 , 
issued to Williams entitled, "Preventative Maintenance and Diagnostic System". 

10 Williams discloses a maintenance system for a machine controlled by a 
programmable logic controller having a memory that stores input data received firom 
the machine and output data communicated to the machine. The maintenance system 
includes a computer interfaced to the memory to be non-intrusive with respect to the 
programmable logic controller, the computer being operative to monitor a state of the 

15 machine over a plurality of manufacturing cycles, including input and output data. 
The computer is able to detect a fault condition for a component of the machine based 
upon a state of a current manufacturing cycle and a state of a previous manufacturing 
cycle. A display device is in conunxmication with the computer to alert an operator 
of the fault condition and to graphically display a location of the component within 

20 the machine to facilitate a corrective measure by the operator. One problem with the 
prior art, however, is that the operator is always tethered to the computer and must 
provide commands through a keyboard or like device. Furthemaore, the maintenance 
instructions are native to the machine that is in need of repair or maintenance, 
requiring that it be in the "on" position during maintenance. Yet another problem 

25 with the invention as described is that the machine in need or repair must have been 
pre-programmed to respond to the maintenance requirements of the hard wire tether 
and be able to communicate with the computer to which the machine is tethered. 

For existing systems in the repair and maintenance industry, or within 
industries in need of highly specialized on-site repair and maintenance personnel, 

30 there is presently no system, apparatus or method of providing multimedia data 
acquishion, interpretation, instruction and reporting. 



wo 00/63763 



PCT/USOO/08019 



4 

SUMMARY OF Hffi INVENTION 

The present invention provides a light, portable hands-off or hands-free 
maintenance and repair system, using a task-specific hypertexted, animated, 
voice-synthesized, and/or voice-activated communications and search engine that is 
able to access a task-specific database. The instructional program or engine is made 
task-specific by accessing a database of machine, process or operational specific data, 
which may be computer data^ audio and even video clips that provide the user with 
real-time instructional information about the task or tasks that the user is to perform 
at a remote location. 

The computer device is sufficiently lightweight to use at the remote location 
and provides hands-free command of the engine and access to the task-specific 
database so that the user may perfom maintenance and repair on the machine, 
process or operation with the use of both hands. The task-specific information may 
include technical views and design diagrams of the machine, process flow or 
operation. The task-specific data will also provide instruction for accessing the 
machine, process or operation and be displayed to the viewer through a display device 
that may be, e.g., a head set with partial view of the operation and of the computer 
display or may even be a hand-held display that the user positions within the user's 
view. 

The present invention includes an information retrieval and input system for 
displaying task-specific data, with a computer having an input/output device, access 
to a large capacity information storage and a retrieval device interfaced with the 
input/output device adapted to receive a memory medium containing the 
predetermined digitized data. The storage and retrieval device is adapted to access 
the task-specific data on the memory medium and to provide the data to the 
input/output device, the predetermined digitized data may be organized in, e.g., a 
hypertext format. 

Hypertext may be accessed directly through a hypertext engine or via software 
written to access hypertext-based data, as well as data in other formats, e.g., in 
Microsoft Access or D-Base format. Audio and video input and output devices may 
be included in a headset adapted to be worn by the user. The head mounted image 
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display is supported by the headset having a virtual image display having one or more 
eye-pieces adapted to be positioned within the view of the user while performing a 
specific task thereby providing video display information from the computer to the 
user in a hands-free, lightweight manner. 
5 The task-specific storage and retrieval instructional program of the present 

invention provides read/write capability for task-specific data, as well as, hands-free 
voice recognition for providing access to data that is task-specific on a storage 
medium or via download. The system provided herein may be part of a network. One 
such system for use with the portable computer of the present invention is a network 

10 card, which provides a wireless computer interface with other users or even a 
customer support center (CSC) connected through a wireless antenna, through a hard- 
vnred computer network, or a combination thereof 

In operation, the user looks through an eyepiece to view a virtual image that 
appears in front of the eyepiece, creating the illusion of a high-resolution computer 

15 screen for view by the user. The user sees an image that a number of menus and 
buttons that the user accesses by voice-conunand without the need for a movement 
controlled device such as a mouse or keyboard. 

The display and input devices are connected to a very lightweight computer 
that accesses data from a large capacity storage device, like a CD-ROM or 

20 DVD-ROM drive and stores user responses on, for example, a memory card. These 
devices may hold the task-specific database(s) that allow the user to make the 
instructional program task-specific simply by exchanging disks at the remote location. 
The CD-ROM or DVD may be left at the remote site, thereby allovmig any SIMON 
user to reach the remote location and access the storage medium to access task- 

25 specific maintenance and repair information. The data may be transmitted to the user 
at the remote location via a hard-v^re or by wireless transmission. The data may even 
be stored on hardware, such as memory chip that contains the task-specific 
information. Other mechanisms for downloading information are also within the 
scope of the invention, including network access, remote access (via telephone lines 

30 or otherwise) or satellite-based communication systems. 
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More particularly, the present invention is directed to a portable instructional 
system that includes a computer sufficiently lightweight and designed so that it may 
be worn by a user to which a memory has been connected. The memory is connect 
to and under the control of the computer. The system also includes a display device 
5 that can receive display signals firom the computer for visual display to the user and 
an input device by which the user enters conunands to the computer. An instructional 
program is provided that the computer accesses and stores in memory in response to 
a user command and displays information concerning a task to be performed by the 
user on the display device in response to commands from the user. 

1 0 In one embodiment of the present invention, the portable instructional system 

that may be worn by a user may also be described as having a computer of a size 
sufficiently small to be carried by a user to which a memory is connected to and under 
control of the computer. A display device that can receive display signals from the 
computer for visual display to the user and an input device by which the user enters 

15 commands to the computer is also provided. The instructional program that the 
computer accesses and stores in memory contains instructional information 
concerning a task to be performed by the user. The system also includes a video 
\ camera of a size sufficiently small to be worn by a user connected to the system that 

transmits images from the camera to a remote site and that is accessed by a 

20 communications system, whereby the user can communicate with an assistant 
viewing the images at the remote site concerning the task being performed. 

The instruction system may include a first computer of a size sufficiently 
small to be worn by a user which is connected to a memory connected to and under 
control of the first computer. A display device is provided that can receive display 

25 signals from the first computer for visual display to the user as is an input device by 
which the user enters conunands to the first computer. The instructional program is 
accessed by the first computer, which the computer stores in memory. The program 
contains instructional information concerning a task to be performed by the user at 
a location. The system also includes a second computer at a remote site from the 

30 user's location for use by an assistant working with the user, which is connected to 
a communications program by which the second computer receives and displays data 
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from the first computer, which data is of a type that aids in the performance of the 
task. The first and second computers communicate through a wireless data 
transmitting and receiving device connected to the first computer, which device can 
transmit data to the station and receive data from the station. A wireless data 
5 transmitting and receiving station having a range, which station can transmit data to 
and receive data from the first computer anywhere with the range is also provided. 
Finally, a communications link is provided by which the second computer can send 
data to and receive data from the wireless station while the user is working on the 
task within range of the wireless station. 

10 A computer communication system for use with the present invention 

includes a computer of a size suflQciently small to be worn by a user to which a 
memory is cormected. A display device that can receive display signals from the 
computer for visual display to the user is connected to the computer as is an input 
device by which the user enters commands to the first computer. Wireless data is 

1 5 transmitted and received through a device connected to the computer, which device 
can transmit data to the station and receive data from the station. The wireless data 
transmitting and receiving station has a range, which station can transmit data to and 
receive data from the first computer anywhere within the range. Also, a computer 
network that can send data to and receive data from the wireless station while the user 

20 is working on the task within range of the wireless station is provided. 

Yet another embodiment of the present invention is an instruction system 
includes an instructional program that controls a computer and is stored in memory, 
which program contains instructional information concerning a task to be performed 
by a user at a location. An input/output program is also provided that controls the 

25 interaction between the computer and a display device and at least one audio input 
and output device. A report generating program under the control of the instructional 
program aids the user in generating and transmitting an electronic report relating to 
the task performed by the user at a location. Finally, an instructional database under 
the control of the instructional program, which database comprises information 

30 specific to the task to be performed by a user at a location is provided along with a 
communications program by which the computer conununicates data from the 
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computer to a computer network that can receive and transmit data, which data is of 
a type that aids in reporting performance of the task. 

Another embodiment of the present invention is a a computer communication 
system that includes a first computer of a size suflBciently small to be worn by a user 
5 that is connected to and control a memory and a display and audio device that can 
receive display signals from the first computer for visual display to the user and 
through which the user may enters commands to the first computer. The first 
computer is connected to a wireless data transmitting and receiving device, which 
device can transmit digital data to a wireless data receiving and transmitting station, 

10 which station is capable of and receiving and transmitting data fix)m the station. A 
second computer of a size sufficiently small to be worn by a user is also able to 
transmit and receive wireless data and allows the users of the first and second 
computers to communicate in real-time. The second computer is also connected to 
and controls a memory, and a input and display device. 

15 The present invention also includes a method of providing a user with an 

instruction system including the steps of providing a user a computer sufficiently 
lightweight and designed to be worn by the user, connecting a memory to, and under 
the control of, the computer, displaying on a display device display signals fix^m the 
computer within the visual range of the user, providing an input device by which the 

20 user enters commands into the computer and accessing an instructional program that 
the computer stores in memory in response to a user command and displays 
information concerning a task to be performed by the user on the display device in 
response to commands firom the user. 



25 
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B RIEF PgSC RIPTION OF TH E PRAWINQg 

For a more complete understanding of the features and advantages of the 
present invention, reference is now made to the detailed description of the invention 
along with the accompanying figures in which corresponding numerals in the 
different figures refer to corresponding parts and in which: 

Figure 1 is a diagram of the basic components of the system disclosed herein; 

Figure 2 is flow chart of the interactive components or modules of the SIMON 
engine and system as they relate to a user performing a task; 

Figure 3 is a first flow chart of a basic operating system and main appUcation 
of the present invention; 

Figure 4 is a second flow chart of the main application and its relation to 
various modules; 

Figure 5 is a flow chart of the maintenance procediues module; 

Figure 6 is a flow chart of the help module; 

Figure 7 is a flow chart of the parts search module of the invention; 

Figure 8 is a flow chart of the part number locator and display system; 

Figure 9 is a flow chart of a basic reports module for use with the invention; 

Figure 10 is a flow chart of the security and administrative module; 

Figure 1 1 is a flow chart of the communications module for connecting to 
peripherals that cormect the present invention with outside sources of commxmication; 
and 

Figure 12 is a flow chart of the diagnostics module for use with one 
embodiment of the present invention. 
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DETAILED PESCWPTIQN OF THE INVENTION 
While the making and using of various embodiments of the present invention 
are discussed in detail below, it should be appreciated that the present invention 
provides many applicable inventive concepts which can be embodied in a wide 
5 variety of specific contexts. The specific embodiments discxissed herein are merely 
illustrative of specific ways to make and use the invention and do not delimit the 
scope of the invention. 

The present invention is directed to a system, apparatus and method of 
providing real-time, on-line help to users that may need assistance in operating, 
10 repairing or maintaining a complex system. The present invention is not only a 
complete, interactive, learning repair and maintenance system, it is customizable for 
the individual machine that is in need of operator intervention. 

Known as SIMON, for System Integrated Maintenance Online, the present 
invention provides the customer or user with a complete online maintenance package 
15 for their specific system. SIMON enables the user to view pages written in, e.g., 
HyperText Markup Langxiage (HTML). Maintenance procedures are written in 
HTML. Context sensitive help is available to the user regarding the operations of the 
SIMON engine. A parts search for the customer's parts database is also available to 
the user regarding the operations of the SIMON engine, A parts locator, which 
20 enables a user to locate a specific part, is also available. SIMON also provide reports, 
security, communications and learning modules. 

Each customer that uses the SIMON engine for a particular machine, 
operation or process determines which modules they vnl\ need, and only need to 
purchase or use the modules that they need. The content of the SIMON system is 
25 specific to the customer's specification equipment, version, machine process or 
operation. SIMON recognizes voice, mouse, keyboard and touch screen input, and 
may also use the default windows screen data. 

One type of computer system that may be used with the SIMON system is 
described in United States Letters Patent 5,844,824, issued to Newman, et al., the 
30 relevant portions of which are incorporated herein by reference. Newman discloses 
a portion of a hands-firee, portable computer and system that may be used with the 
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present invention. The body-worn or portable system includes a hands-firee computer 
system. The system does not rely upon a keyboard input or activation apparatus but 
rather has various activation means, which are generally hands-free. 

The system of the present invention may be used with other systems, other 
5 system components and communication apparatus as described herein. Also, various 
components of the present system may be body worn or placed in a discoimected 
location if desired. 

Another such computer system is disclosed in United States Letter Patent 
5,581,492, entitled "Flexible Wearable Computer", issued to Janik. The relevant 

10 portions of the Janik patent are incorporated herein by reference. Janik discloses a 
flexible wearable computer in the form of a belt that combines a computer that 
includes a microprocessor module, a RAM-I/0 module, memory modules, a power 
supply module, and bus termination modules that are coimected with flexible signal 
wires. The computing elements are physically attached to a flexible non-stretchable 

1 5 belt or harness, and may also include a protective cover. For example, the computer 
may be powered by an 8 hour six volt power supply. The power supply may be a 
modular battery that is worn as part of a harness or on a belt. The flexible 
non-stretchable wearable belt or hamess may be secured around various parts of the 
body. The input and output device or devices are connected to the flexible wearable 

20 computer by the I/O bus attached to the output and mput ports. 

The portable computer hardware for use with the present invention may be 
implemented in many different ways. For example, each module may form part of 
a solid thermoset plastic casing. Alternatively, a two-part hard plastic shell may be 
used to encase a mother or one or more sister-boards having thereon the computer 

25 components, input/output devices, control devices and the like. 

The components and support circuitry needed will generally be surface 
mounted based on size and soldered to a mother or sister-board. The components 
may, alternatively, be affixed to the circuit board with a conductive epoxy or other 
conductive plastic. The computer may be made even thiimer and more flexible using 

30 chip-on-board and or chip-on-chip manufacturing technology. Each integrated circuit 
may be bonded directly to a small printed circuit board and the terminals may be 
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electrically connected to the board. Each mtegrated circuit may also be covered with 
a heat radiating element to reduce heating of the integrated circuit and increase heat 
dissipation. 

The computer may be implemented as one long multilayer polyamide flexible, 
5 or rigid-flex, printed circuit board. As an entirely flexible printed circuit board, the 
modules that form part of the computer system increase the rigidity and durability of 
the portions of the computer with electronic components. 

To interconnect within a computer component or among the different 
integrated circuit components or even from one components (See Figure 1, e.g., the 

1 0 computer 1 02) to a separate component of the flexible portable computer (See Figure 
1, e.g., storage medium 122) flexible metallic wires or even non-metallic electrically 
conductive filaments or wires may be used. The same holds true for the connections 
between the computer hardware and the communications devices and/or the head gear 
and commimications devices thereon. The computer may even be implemented using 

1 5 fiber optic device connections. For such a connection, optical fiber filaments instead 
of metallic or non-metallic conductors are used with photosensitive detectors at each 
end of the filament that commimicate the optical signal. One advantage or flexible 
wires or optical connections is that they are easily replaced when worn or may even 
be disconnected and connected to another device, such as another head gear or even 

20 a stand-alone computer. 

Furthermore, the configuration of the flexible portable computer need not be 
in a belt but may be part of a harness, thereby causing a better distribution of the 
weight of the components during use. The module and bus assembly may be 
fashioned in a variety of ways such as a vest. When worn as a vest, the harness that 

25 contains or holds the flexible computer may even be adapted to look like normal 
uniforms or clothing, thereby increasing comfort. 

Input and output devices for use with the flexible computer may include, for 
example, a touch sensitive flexible liquid crystal display (LCD) hanging firom a belt 
or even worn on a forearm or thigh. The wearer controls the computer by touching 

30 virtual graphical elements on an LCD with either the right or left hand. An added 
advantage of this design is that individuals that are, for one reason or another. 



1 
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physically challenged may benefit 6com the availabiUty of a fiilly integrated hands-ofif 
or even hands-on flexible portable computer. A pen-based input and output device 
may also be used, in combination with, e.g., a Palm Pilot or other hand-held device 
that may interface with the portable computer and the SIMON engine. For example, 
5 the pen location data is sent to the computer via an infira-red link or hard connection 
and the computer sends corresponding graphical information to the pen-based input 
and output device. 

Another possible configuration for the portable computer for use with the 
invention is using a fi«e-space pointer input device and a private eye heads-up display 

10 as the output device. A firee-space pointer, such as a GyroPoint, translates relative 
three-dimensional motion of the hand into digital pulses that are monitored by the 
computer. Software drivers translate the digital pulses into corresponding movements 
of the cursor in the virtual screen generated by the heads-up display. Both the 
firee-space pointer and the heads-up display may be connected to the computer via 

1 5 tethers, thin wires or optically. 

The present invention also includes connections via wireless transmission to 
a Local Area Network (LAN) or a Wide Area Network (WAN). A wireless 
transceiver, e.g., a WaveLan® wireless network IEEE 802.11 card (Lucent 
Technologies), may be fiinctionally coimected to the computer to provide for digital 

20 communications to a wireless network backbone. The transceiver commimicates via 
radio-fi*equency pulses, which may also incorporate echo handling capabilities for 
delay spread. One or more wireless repeaters and or transceivers may be moxmted 
over the work environment in which the portable computer is worn and used. 
Security features may also be included in the wireless commimication such as data 

25 encryption. 

Wireless communication may also be via infi^-red signals, however, this 
method generally requires line-of-sight contact between the receiver/transceiver and 
the antenna or detector on the floor, wall or ceiling. Existing LANs or WANs may 
be easily retrofitted to add wireless communication capabilities without the need for 

30 rewiring. 
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Also, the wireless communication may allow SIMON users to communicate 
peer to peer during repair and maintenance operations directly or through a network. 
With the addition of a LAN/WAN based outdoor antenna, the present system may 
even be used to communicate SIMON users in two separate buildings or even in 
5 remote locations. The LAN/WAN communications network connection may also be 
used to connect to an intranet or the internet to communicate with remote sites within 
. a same group or corporation to commimicate to a centralized consumer support center 
(CSC). 

An audio output device may include one or more earphones and an audio 

10 input device a microphone, such as a headset. Real-time text to speech conversion 
may be provided using commercially available voice-recognition software, e.g., 
NaturallySpeaking. The microphone is adapted to receive speech data from the 
computer and to provide audio input to the earphone. The microphone may even be 
a lapel wom microphone such as an Electret microphone from Gentex. A light 

1 5 weight headset having an earphone, microphone and a video display may be provided 
in combination and connected to the computer. 

Figure 1 shows the components and features of a portable computer 100 for 
use with the present invention as it is wom by a user. A computer 102 may include 
a housing and be secured by a belt, harness or other attachment apparatus, flexible or 

20 not (in the embodiment depicted a strap or belt 1 04 which is wom around the user's 
1 06 waist for securing the system unit or housing 1 02), to a user 1 06. The computer 
102 further includes a monocular or duo-ocular display unit 108 for receiving 
information from the computer 102. The display unit 108 displays information 
received for the computer 102 xmder the control of user 106 or a remote operator. 

25 The display unit 108 may include a headband 1 10, a display screen 1 12, a speaker 
1 14, and a microphone 116 connected to the headband 1 10. The headband 1 10 is 
adapted to be wom by the user 106 in any convenient location, but preferably upon 
the user's 106 forehead, as depicted. The position of the display screen 112 may be 
adjusted via the adjustable arm 118 so that the operator may view comfortably 

30 information displayed on the display screen 1 12. As depicted, the adjustable arm 
1 1 8, is in fact, not only an adjustable arm but a camera that displays to the user 106 
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through display screen 1 12 or to an operator at a remote location the image before the 
user 106. The display screen 1 12 is electrically connected to the computer 102, in the 
present embodiment, via a cable 120, although other ways of connecting the two 
components may alternatively be employed. 
5 The computer 102 may further include transducers and converters to provide 

conununications with display unit 108 for receiving activating (from audio, 
eye-tracking or EEG) commands from the user, for converting the receiver commands 
into electrical signals, for recognizing the converted electrical signals and for sending 
the recognized electrical signals to a processor within the computer 102. In the 

10 present embodiment, the transducers and converters may includes the microphone 
1 16 through which the user 106 may issue verbal commands to the system, apparatus 
and method of the present invention. The microphone 1 1 6, the speaker 1 14 and the 
display 1 18, in the present embodiment, are electrically-connected to the system imit 
106 via a cable 120, although those with ordinary skill in the art will appreciate that 

1 5 any input or transducer device could be used and that the input or transducer could 
be supported by the user at some other location. 

The computer 102, in the present embodiment, may fiirther include physical 
or non-physical connections to provide commimication between the computer 102 
and other peripherals 122 that communicate with the computer 102 and that may 

20 provide, for example, read-only or read-write memory or other capability (such hard 
disk, CD-ROM, DVD, Magneto-optical, or like storage devices). In the embodiment 
depicted, the peripheral pack 122 is also attached to the belt 104 and is electrically 
connectable via cables 126 to the computer 102. While the present invention as 
illustrated in Figure 1 is depicted using voice activation, it will be understood by 

25 those of skill in the art that other ways of connecting, accessing and controlling the 
computer 102 may be substituted therefor as described herein. 

Also depicted in Figure 1 , is a wireless remote communications node 128 that 
serves to intercoimect the user 106 vnth a remote wireless network 140. A wireless 
antenna or receiver 142 may be positioned in the same or an adjacent room, 

30 depending on the range of the wireless remote commimications node 128 and the 
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reception and signal throughput required for transmission of audio, video, computer 
data, or any combinations thereof. 

The antenna 1 42 may connect through a local area network (LAN) or a wide 
area network (WAN) through, e.g., a hard wire connection 144, to a central server 
5 146. The central server may be located in the same facility or adjacent facilities that 
are connected through the LAN or WAN to the remote wireless network. 
Alternatively, or as part of the same system, a connection 148 to an intranet, the 
internet, via modem or other such digital or analog conununications system may be 
used to remotely transmit a signal to a remote location. 

1 0 Figure 2 shows a chart with the operational access that SIMON provides to 

the user of the SIMON portable computer and system. The SIMON engine, 
designated as 200 provides the user 202 access to a variety of functionalities. The 
SIMON engine 220 is able to access one or more modules, in real-time, to provide 
the user 202 with information as to the status of the main application 204 and, e.g., 

15 a help menu 208 for running the main program or any of the one or more modules 
that serve the user 202. For example, the user performing a specific task at a remote 
location may access the maintenance procedures module 206 to obtain step by step 
instruction and maintenance procedures for repairing a particular piece of equipment 
or for the use of certain tools. The SIMON engine 200 also allows a user that is in 

20 the maintenance procedures module 206 to access either the parts search module 2 1 0 
or the parts locator module 212, which permit the user to find out parts numbers and 
availability in the parts inventory, locally or globally. 

The SIMON engine 200 is also able to interface with a reports module 214 
that is capable of taking the maintenance procedure information derived fix)m the 

25 maintenance procedure module 206 and submit a report, in real-time, on-line or in a 
subsequent data dump. The report may be used not only to report that the repair has 
been completed, but may also be capable of providing fiiU parts listings for any new 
parts used or needed. By tying the reports module 214 into a LAN or WAN, the 
report may be used to standardize reporting and maintenance procedures without the 

30 vagaries and informality often found with hand-written maintenance records. The 
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reports module 212 itself, or a compatible application, may also provide a docket of 
maintenance to be conducted or that has been missed to update maintenance records. 

The SIMON engine 200 is also capable of accessing security protocols that 
permit or limit the user's 202 access to the SIMON engine 200 itself, for 
5 commimications outside of the SIMON engine 200, for access to information firom 
the other modules, or any other security protocols that may be necessary. The 
SIMON engine 200 is also capable of improving its help to the user 202 through the 
learning module 218. The learning module 218 permits the SIMON engine 200 to 
increase user 202 performance by, e.g., noticing patterns in the user's 202 method of 

1 0 accessing the SIMON engine 200 and its modules to anticipate the user's 202 needs. 
The diagnose module 220 allov^^ the SIMON engine 200 to provide troubleshooting 
information to the user 202 with real-time or on-line access to diagnostic protocols, 
equipment or remote diagnosis capabilities. 

The communications module 222 allows the SIMON engine 200 to interface 

15 with a variety of sources of information, communications, data, and the like. For 
example, the commimications module provides not only the voice-recognition 
capabilities to the SIMON engine 200, but also allows commimications within the 
equipment that serves, and is in contact with the SIMON engine 200 and the user 202. 
The communications module 222 is capable of accessing audio and video captured 

20 on any storage medium associated, or in communications with, the SIMON engine 
200. The communications module 222 also permits the SIMON engine 200 to 
communicate by hard wire or wireless with a LAN, WAN, intranet, intemet or other 
network with which the SIMON engine 200 may need commimication. Data transfer 
through these local or remote networks may include, e.g., data, sound, voice, images, 

25 video or other analog or digital information that the user 202 may be able to access 
through the SIMON engine 200. 

More importantly, the SIMON engine 200 provides a "shell" program that is 
used to access machine, process or operation specific instructional progranmiing, 
databases, troubleshooting, reporting and other information to a user performing a 

30 task at a remote location. The instructional task specific may be stored at the remote 
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location in the form of a, e.g., CD-ROM that contains data, audio, image and video 
information that is specific to the task before the user. 

If a user must repair or maintain several different pieces of equipment in a 
given day, e.g., a single SIMON engine loaded on a portable computer is able to 
5 provide the user with task-specific information by loading the task-specific data. 
Once the task has been conducted, the task-specific data is removed or erased and 
new task-specific instructional data is accessed through the SIMON engine at a new 
location or for a different machine, process or operation. The SIMON engine may 
be implemented in software, in hardware as a ROM or a combination of hardware and 
10 software. 

Main Application 

The main application module 204 of the SIMON engine may have, itself, a 
modular design and may be developed using Microsoft Visual Basic 6.0 or Microsoft 

1 5 Visual C-H- 6.0. The main application modxxle 204 interfaces with a browser engine 
developed with, e.g., Microsoft Visual C-H- 6.0. The main application module 204 
enables the user to view pages written in, e.g., HyperText Markup Language (HTML) 
and interface with a machine's diagnostics file. Support for JavaScript and standard 
web related plug-ins (such as audio, video), may also be included as part of the 

20 browser interface. 

Additional features may include voice activation using AT&T Watson Voice 
Control, or other voice control software and a Computer Aided Design (CAD) 
drawing viewer supported by the Autodesk Whip plug-in. The application may be 
designed to run under Windows95/98, Macintosh or Windows NT operating system 

25 (OS). 

Voice activation allows the user hands-fi"ee operation. All functions achieved 
via buttons or mouse events are capable of being achieved through voice commands. 
The Whip plug-in allows the user to view, e.g., CAD drawings in Drawing Web 
Format (DWF). The fiinctions of zoom, pan, and scroll are controlled by Whip plug- 
30 in, and may be voice activated through voice control software. 
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The main application module 204 may have text-defined buttons across the 
top part of the screen that provides access to the other modules. Main buttons display 
a sub-button menu for additional options. A second set of buttons is displayed along 
the bottom of the screen that will show the general controls most needed for a specific 
5 function. The main application modiile 204 may be designed to be the same for each 
customer or for each user. Some features and properties of the main application 
module 204, however, may be deactivated at the customer's request prior to 
installation, and the customer may even control some features. Commands for voice 
activation may also be standardized, with additional commands customized to meet 

10 customer needs. The standard commands are single or two word "keywords" to 
insure consistency and connection accuracy. 

The response time using a keyboard, mouse, or voice activation to access the 
main application modiJe 204 and the other modules will generally be less than 2 
seconds. Requests for customized commands may be analyzed for complexity and 

1 5 structured to meet the response time criteria or an individual system or user. 

The mam application module 204 of the SIMON engine fimctions as the main 
Graphical User Interface (GUI) firom which all other modules are called. Since 
SIMON is primarily an HTML page viewer, it is visible at all times and will have a 
window area large enough to accommodate the HTML contents, but may be written 

20 in other like codes having like functionality. Each function of the main application 
module 204 is capable of user activation by means of a keyboard, mouse, or voice 
activation. Voice commands may match the button command or common commands 
identified in the command dictionary. Voice cormnanded words that are active within 
a specific HTML page are made known to the user through text variation of the voice 

25 active word. 

Turning now to Figure 3, the SIMON engine may automatically open when 
the user powers on the computer loaded with SIMON. If the SIMON engine is not 
part of the start-up of the computer when the computer was last shutdown, an icon is 
made available on the desktop that will allow the user to use voice, mouse or 

30 keyboard to activate SIMON. SIMON opens with a login prompt 300 requesting the 
user to input their assigned ID and password. The user inputs their login information 
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via a keyboard, voice activated or other input and SIMON determines is the user login 
is valid at 302. On confirmation, the user is directed to, e.g., a splash screen 304 
followed by a display start up screen 306. The option to tour the functions of SIMON 
are selected at decision box 308, which permits the user to start SIMON 3 1 0 or enter 
5 a SIMON tour 312. 

If the login is invalid, an error message 314 is presented to the user and the 
user is allowed to try to re-enter the login name and password at decision box 316. 
If three failed attempts are made, then the user is forced to exit the application at 3 1 8. 
Additional security may be added that requires administrator intervention to allow 

10 renewed access to the login prompt 300, if necessary. 

After confirmation and log-in, and following a brief change in screen 
appearance or splash screen 304, the user is given the option to tour 3 1 2 the functions 
of SIMON at the main screen display 320. If tour is chosen at 312, the user is 
presented with a training presentation on the basic operation of SIMON. The 

15 presentation will show the user the basic voice commands and how the various 
fimctions work. Upon completion, the user should have a good general understanding 
of how SIMON works and how to access its modules. The user is returned to the 
select option decision box 308, which permits the user to start SIMON 3 1 0 or exit the 
SIMON appHcation 318. The decision for which module of SIMON the user selects 

20 is made at decision box 322, which permits the user to enter a nimiber of SIMON 
functions, including, for example, the SIMON program setup 324, a maintenance 
procedure module 326, a diagnosis module 328, a parts module 330, a reports module 
332, a communications or connection module 334 or a help module 336. Similar 
modules are depicted in Figure 2. 

25 Turning now to Figure 4, in conjunction with Figure 3, if the set-up option is 

selected 324, the user is allowed to choose from a number of options at 400, One 
option for the SIMON engine is to permit the user to select from a number of systems 
in need of maintenance to choose from. At system select 402, the user is prompted 
to select from a list of available systems by displaying a list of machine types and 

30 machine model numbers that are available to the SIMON engine. The user then 
selects from the list of machines displayed 404. Alternatively, the user may be 
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allowed to search by entering a machine to machine model from which to choose or 
that may be downloaded from a local or remote site. Alternatively, the administrator 
may deactivate this feature if the SIMON engine is abready machine, process or 
operation specific. Once the user selects from the list, SIMON will open the SIMON 
5 introductory page and load the appropriate HTML page for that machine and model. 
The user may then choose to start the maintenance operation by indicating that they 
are familiar with SIMON and have knowledge of how to work on that particular 
system in need of maintenance. Alternatively, the user may access, through SIMON, 
local or online reference material and are immediately prompted with a selection 

10 criteria for use and operation. Navigation may be through a keyboard, mouse or 
preferably through voice or visual recognition commands. 

The set-up option menu 324 in addition to allowing access to preselected 
options at 402, also permits the user to access specific files by accessing a drive and 
directory where SIMON data files, such as HTML and database files are located 408. 

1 5 The particular type of machine(s) or the operation(s) that wiU be accessed through the 
SIMON engine may be selected from the menu or directory and access to the same 
may be restricted by the administrator depending on the user. The user may either 
select from a list of file types or may search via name, number or other identifier xmtil 
a match is found. 

20 The set-up display menu 324 also permits the user to access a computer 

support center (CSC) link 410, through which the user may have direct access to one 
or more CSCs. A preselected list may be made available to the user depending on the 
machine and model selected at 402. The configuration options for the CSC may also 
be selected at 412, which will conform to the CSC that is being accessed and the 

25 various options available to the user in terms of data, audio and video access. The 
user may also select set-up options local to the portable system that the user is using. 
One such option is the speech set-up 414, in which the user is permitted to change the 
user speech profile. This option is particularly usefiil when more that one user, or 
even a new user, may be using a portable SIMON system and the computer needs to 

30 be instructed on the particular speech pattern of the user. 
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Likewise, the set-up display menu 324 may be used to access the security set- 
up 418, which permits the user, depending on their level of user, to modify, e.g., user 
accounts and privileges 420. The user is also allowed to close 422, exit 426 or shut 
down 430 the system. The close option 422 closes the main set-up window 424 and 
5 returns the xiser to the start-up display screen 306 (see figure 3). The exit option 426, 
exits the SIMON application all together and returns the user to the desktop of the 
system. The shutdown option 430, on the other hand, exits both the SIMON engine 
or application and initiates or prompts for the operating system of the computer on 
which SIMON is operating to shut-down. 

10 

All user inputs may be activated by, e.g., voice, keyboard, or mouse. Input 
choice may be activated firom a button menu. When a specific button from the button 
menu is chosen, the button sub-buttons are displayed on the screen. The following 
is a brief list of commands that may be automatically accessed by the SIMON engine 
15 by pressing a button on the screen or via single keyword, sound or visual cue from the 
SIMON engine by the user: 

File - File button, when activated displays "File" sub-buttons on screen with 
the following options: 

Open - If "Open" is chosen, the user is given a choice of which HTML file 
20 to open. If only one HTML file is available, that file is the default HTML file to 
open. 

Find ~ "Find" will activate a pop up window allowing the user to speak or 
type the word they wish to find. "Find" will search the active HTML page for the 
requested word and if found will highlight the word. The user is able to close "Find" 
25 and continue in the active HTML page or elect to find the next instance of the word 
by using the "Find Next" button. If no match of the word is found, the user will 
receive a message box indicating "No match found". , 

Exit - If "Exit" is chosen, the user is prompted to confirm close with a 
Confirm Exit box. If the user chooses No, the Confirm Exit box will close and the 
30 user remains in the active HTML page. If the user chooses Yes, SIMON saves any 
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updates to the appropriate report Jog file and then exit the active HTML page. 
SIMON will then return the user to the Windows/NT OS desktop. 

Print - SIMON system may be attached to a printer if the user wishes to 
produce hard copy reports or even to "print" to a remote location electronically as a 
fax or data transmission. If the user selects print, a SIMON Print box will appear to 
identify the default printer and allow the user a print selection of reports or HTML 
page. The daily report log is highlighted as the default print selection. Date defaults 
to current system date for daily reports. If the user selects any other item, that item 
becomes the highlighted selection. The user may select an alternate date to generate 
a desired report. The HTML page selection refers to the active HTML page. 
Choosing "Ok" will print the highlighted selection to the designated printer. Cancel 
will close print and SIMON will retum to the active HTML page. Individual report 
specifications identified imder the Report Module, as described in detail hereinbelow. 

Security - If "Security" is chosen, a security log-in window is presented. 
On verification of the users ID and password as administrator, the user is allowed to 
add users, modify existing users and delete existing users delete the report log-files 
and deactivate the machine type entry box through the "Administration Edit" screen. 
The "Machine Type Input Box" default is activated. If deactivate is chosen, the 
button will change to "Activate Machine Type." 

Shutdown — If "Shutdown" is chosen, the user is prompted to confirm 
shutdown with a "Confirm Shutdown" box. If user chooses No, the "Confirm 
Shutdown" box will close and the user will remain in the active HTML page. If the 
user chooses Yes, SIMON will exit the active HTML page then proceed to initiate the 
complete OS shutdown sequence. "Shutdown" allows the user to turn the power off 
after shutdown complete, without having to initiate shutdown through the 
Wmdows/NT OS. 

Diagnose — Diagnose allows the user to connect to the specified machine 
diagnostics file. Diagnose activates the Diagnostics Module as identified in the 
Diagnostics Module, as described in detail hereinbelow. 
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Parts - Parts when activated displays Parts sub-buttons on screen. Parts 
provides access to the task specific parts database and parts locator through the parts 
module. 

Search — Search activates parts_search. Parts_search allows the user to 
5 search a task specific database for a part by using a part number or part description. 
Parts__search is defined in the Parts Module specifications- 
Locator ~ Locator activates partsjocator. Parts_locator allows the user to 
display the customer defined, task specific graphical view of the equipment parts. 
Parts_locator is defined in the Parts Locator Module specifications. 
10 Reports - Reports when activated displays Reports sub-buttons on screen. 

Reports are defined in the Reports Module and are also task specific and may be 
customer defined. Reports allow the user to choose the type of report to print. To 
insure security control of information, only the user with appropriate administrative 
rights is allowed to delete report_log file information. 
15 Daily - TheDaily report is the defeult report that is generated when the user 

logs on. This is the default report to print when File jrint is activated. 

Custom - Custom reports will allow the user to generate a report by 
specifying the time interval desired. The Custom report also allows the user to create 
and filter a report by type of maintenance (repair or preventive maintenance). 
20 Connect — Connect when activated displays Coimect sub-button on screen. 

Coimect allows the user to establish a connection with a CSC, a network or even 
another SIMON user. The connection allows full duplex audio and live or still image 
video transmission, as needed. 

CSC - If the user selects CSC, SIMON will deactivate the voice command 
25 features of the HTML application and SIMON interface. Once voice commands are 
deactivated, SIMON will establish an audio/video connection allowing the user to 
have fidl duplex net conferencing type communications with the default CSC. 

Cormection firom a SIMON user performing a task to the CSC may be 
provided by, e.g., connecting the SIMON user having a Wavelan® wireless 
30 transceiver to a server having a wavepoint connection. The wavepoint hardware and 
software connect to a VBX telephone box-CODEC hybrid connection. The 
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VBX/CODEC transmits a SIMON inquiry through an ISDN connection to another 
XBC/CODECLAN connection at the CSC. The CSC can then receive, transmit and 
display information from the SIMON xiser, be it data (such as a report or system 
status), video and audio that allow the personnel at the CSC to view the task that the 
5 SIMON user has an inquiry about in real time. Of particular importance is video 
transmission in color, where the color of wiring may be of critical importance to 
completing the task, but more importantly, for the safety of the user in particularly 
dangerous situations. 

The audio connection may be active at all times during the CSC connection. 

10 The video connection is at the control of the user and activated as needed. If the 
video connection is activated, a small window will open on the user screen allowing 
the user to view the video feed to the CSC. 

When the user has completed the communication with the CSC, the salutation 
"good-bye", will trigger SIMON to disconnect and close the CSC connection. 

1 5 SIMON will verify that the comiection is closed and re-activate the voice command 
features of the HTML file and SIMON interface. 

The customer will define the default CSC. If the customer chooses to have 
additional CSC listings, each CSC will have its own unique identification name and 
will follow the same series of commands as if selecting the default CSC. 

20 Help - Help when activated displays Help sub-buttons on screen. The user 

can select Help Contents or About SIMON. 

Help Contents — Help Contents opens a new window and allows the user 
to choose the type of help wanted. The choices for help imder Help Contents are 
SIMON Commands, Parts Search, Viewing DWF Drawings, About Reports, About 

25 AudioA^ideo and Dictionary. Each of these help topics will offer definitions or 
instructions depending on the title chosen. Help Contents is defined in Help Module, 
About SIMON - About SIMON opens a floating window. About SIMON 
contains the legal information, customer registration information, license infonnation 
and plug-in information that are required to use SIMON. About SIMON is defined 

30 in the Help Module. 
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The SIMON engine may also have a failure contingencies an error message 
is displayed if data is not accessible. The user is able to connect to a CSC, The Main 
Application program is written in Microsoft Visual Basic 6.0 and/or Microsoft Visual 
C-H- 6.0. Computer Security Requirements may include a security level set up in the 
security functions. Human performance requirements, and/or reports thereof, may be 
included for the purpose of monitoring user performance during a particular 
maintenance operation. 

As mentioned hereinabove, individual modules that are part of SIMON are 
used in conjunction with the HTML content and within its context, that is depending 
on the machine and model. The machine/model specific modules may contain 
program default information as well as customized information per customer request. 
For example, default information are items that are part of the SIMON application 
such as the command dictionary within the help module 208. If the user requires 
assistance from a customer service center (CSC), they are able to connect and send 
both audio and video by means of net conferencing via the communications module 
222. Upon speaking the command for AN connection, SIMON will dial the 
customer service center and activate the video camera for live feed. Commands (e.g., 
mute or pause) for the AA^ will give the user some control over the functions of both 
the audio and video feed. 

Upon completion of using SIMON, information pertaining to the specific 
machine/model and repair parameters that are preselected or that are modified in the 
set-up module may be automatically saved in, e.g., a reportjog file for future 
reference and or report output for use in the reports module 2 1 4. Before completion 
of log-off, the user is prompted to verify the type of maintenance or repair performed, 
if the repair/maintenance task was completed, and add any additional comments 
regarding the repair/maintenance the user may wish to add. A standard set of report 
options as well as a set of customized reports may be provided for the specific task 
and may be reported as defined in the Report Module 214, described hereinbelow. 
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The user will have the choice of "Shutdown" or "Exit". Choosing 
"Shutdown" instructs SIMON to initiate a complete shut down of the system. 
Complete shutdown from SIMON will activate the Windows/NT OS shutdown 
sequence and allow the user to turn off the power when prompted to do so. "Exit" 
5 closes SIMON and returns the user to the Windows/NT desktop allowing the user to 
initiate shut down when ready. The user may be required to enter the machine type 
and machine number of the machine they are working on (this feature can be 
deactivated by administrator), 

If shutdown is chosen then the entire SIMON engine program may be shut 

1 0 down by connectmg into the SIMON-operating system automatic shutdown protocol 
prior to complete program shutdown. The user may also chose to exit the cxirrent 
customized machine or operation module and return to the main SIMON screen. A 
default set of voice command may also be activated through the use of a keyboard or 
mouse. Additional commands may be added at customer request. Audio and video 

1 5 net conferencing, auto dial capabilities may be provided when connection to a CSC 
is requested. A video stream feed allows a CSC the same view as user, while a full 
duplex audio connection allows the user to speak to the CSC direct. 

Maintenance Procedures Module 

20 Specific performance requirements for the maintenance module 206 may be 

provided using information displayed using HTML (HyperText Markup Language). 
Button options allow the user to move between procedures. The maintenance 
procedures will allow the user to view online procedures to repair, maintain, clean, 
etc. Video (MPEG -Motion Picture Experts Group- files) are viewable from the 

25 maintenance procedures. The content is task specific, which may be selected in the 
set-up menu. 

Figure 5 provides a flow chart of the maintenance procedure module 326. For 
the maintenance or repair procedure data and information to be accurate, the user may 
be prompted to select the required machine, process or operation type and number; 
30 if one has not already been selected. Next, the user is prompted to select a 
maintenance operation option at 500. Two options that may be made available to the 
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user are to instruct SIMON to provide maintenance information related to procedures 
that are either unscheduled 502 or preventive 504. Depending on the type of 
maintenance operation that is selected, the SIMON engine will display a browser 
window that permits for navigation of the appropriate HTML menu page 506. The 
5 browser window provides the user with options to select at 508, which permit the 
user to access, for example, hyperlink controls may also have standard and 
customized browser controls for further navigation. The user may even be provided 
with a list and schedule for each of the maintenance operations, whether preventive 
or scheduled. 

10 If hyperlink 5 1 0 is chosen, then a context sensitive display 5 1 2 is activated 

that provides the user v^th specific information about the maintenance procedures. 
For example, MPEG video, Autocad drawings, viewer and schematics, etc., are 
displayed for the user. Altematively, the user may select to enter a browser control 
514 window though which the user select may elect to operate the functions of the 

15 browser, such as home, refresh, back, print, find, etc., that are enabled through the 
HTML menu page 506. 

One of the options provided at the maintenance procedure options prompt is 
the display of the graphics, text and any links that are provided for that particular 
option. An option display may, for example, include a link to the context sensitive 

20 display At the option display the user is allowed to access information that is selected 
by paging down or up through the HTML page for the maintenance procedure(s). 
Altematively, and if available, the user may be prompted to display a sound or video 
clip, or may simply return to the top of the page The user may be provided with the 
source data and/or reference equipment or permitted to input context specific 

25 information at the administration level, if appropriate. 

HTML files will lead the user through the maintenance procedures, by hyper- 
linking with point and click or voice activation. The maintenance procedure HTML 
displays may display the procedure using graphics, text, audio or video. The 
information contained in the HTML is customer specific data. The user can choose 

30 a "Show Me Hov/' option to view video (MPEG files) for tasks requiring supporting 
motion video. 
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As with the previoias module, and in fact all the modules of the SIMON 
engine, these are voice, visxially, keyboard or mouse activated. The user will select 
button options to view the maintenance procedure data, as follows. The content is 
specific to the user's task and is displayed as follows. The user is able to view video 
5 clips by clicking or speaking the "Show Me How" hyperlink. 

Help Module 

The help module shows the user a number of options that permit access to 
help information from a variety of sources and at a variety of levels. For example, 

10 help button options allow the user to select the type of help requested. Two 
immediate options that may be made available to the user are "Help Contents" and 
"About SIMON". The help information may also be displayed using HTML. The 
help module allows the user to view on-line help information for the various 
functions of SIMON as well as the legal requirements necessary for the ownership 

1 5 and use of SIMON. The HTML help pages will contain text and graphics. 

Specific performance requirements for the maintenance module 206 may be 
provided using information displayed using HTML (HyperText Markup Langxiage). 
Button options allow the user to move between procedures. The maintenance 
procedures will allow the user to view online procedures to repair, maintain, clean, 

20 etc. Video (MPEG -Motion Picture Experts Group- files) are viewable from the 
maintenance procedures. The content is task specific, which may be selected in the 
set-up menu. 

Figure 6 provides a flow chart of the help module 600. First, the user is 
prompted to select a help operation option at 602. Three options are made available 

25 to the user in help module 600 to instruct SIMON to provide help information related 
to the characteristics of the SIMON engine 604, such as the display of copyright and 
version information 606. Depending on the type of engine and version, the SIMON 
engine will display a browser window that contains the appropriate information. The 
user may also select the Tour Simon 608 option, that directs the user into a 

30 multimedia presentation 610 that contains information about the SIMON engine, its 
uses, etc. Alternatively, the user may select the contents 612 menu that directs the 
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user to a browser window that provides the user with options 616 to select at 614, 
which permit the user to access, for example, hyperlink controls 618 may also have 
standard and customized browser controls for further navigation 622. 

If hyperlink 618 is chosen, then a context sensitive display 620 is activated 
5 that provides the user with specific information about the content of help available. 
For example, MPEG video, Autocad drawings, viewer and schematics, etc. are 
displayed for the user. Alternatively, the user may select to enter a browser control 
622 window though which the user select may elect to operate the functions of the 
browser, such as home, refi*esh, back, print, find, etc. that are enabled through the 

1 0 HTML menu page 624. 

The help module 600 may have the same help information for all customers 
for a particular machine, process or operation, however, other more user or task 
specific options may be made available to specific users whether by limiting access 
to the information or by not providing information that may be outside the expertise 

15 of the specific user. 

The help module may be written with Microsoft Visual Basic 6.0. Pages for 
Help may be written in HTML. "About SIMON" information may be written in a 
Visual Basic 6.0 vwndow. 

The contents of the help fiinction for use with the SIMON engine may include 

20 topic selections that provide a list of "Help Contents", which the user may access 
through a selection of preselected help topics. The "Help Contents" portion of the 
topic selection index is a list of available help topics fi-om which to choose. Selection 
of a topic display help information about a specific topic may also be reached by 
searching using letters, keywords or phrases as part of the search topic profile of the 

25 help menu. If the searched for topic is found within the selected help contents menu, 
then the SIMON engine takes the user to the specific location of the search 
information sought, or the user may be prompted to another selection after reviewing 
the material presented. Alternatively, the user may not find a topic and an error 
message is displayed and the user is prompted to try again. At any point, the user is 

30 permitted to chose the close help option that returns the user to the main SIMON 
engine window. 
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The default Help HTML page that contains the selected topics and may be 
created using Microsoft Visual Basic 6.0. Voice, keyboard or mouse activates all 
user inputs. When user selects the Help button, the Help sub-buttons are displayed. 
The user will have the choice of selecting "Help Contents" or "About SIMON". 
5 Help Contents Help Contents displays a list of available help topics. The 

Help Contents HTML page displays a list of available topics. Topics v^U include 
SIMON Commands, Online Reference Parts Information, Viewing D WF Drawings, 
About Reports, and About AudioA^ideo. Each topic vnll link to its specific HTML 
page and further expand on the help information available to that topic. 

10 About SIMON - About SIMON opens a floating window. About SIMON 

will contain the legal information, customer registration information, license 
information and plug-in information that are required to use SIMON. If the Help 
Content HTML page fails to load, an error message will appear. The user will have 
the option to try again or leave the Help Content. 

15 An error message is displayed if the HTML page fails to load. The user is 

given the choice quit or try again. If it fails a second time, the user is instructed on 
how to check for the HTML help page and how to reload the page if necessary. 
Content for Help may be written in HTML and the display configuration will depend 
on the system, operator, machine or operation to be imdertaken. 

20 The Help module may or may not include any fiinctions that may be changed 

by the user. The security may be defined at the programming level or by limiting or 
not providing the user access to help data beyond the preselected criteria of the 
SIMON engine that was customized for that particular use. The help module may 
even provide a dictionary of components and provide the user with alternative 

25 references for a like item in a task specific manner or context. Likewise, the help 
module may provide the user with help as to the use of SIMON, its command 
structure and capabilities. The help module may even be used to obtain step by step 
instruction for communication v^th a CSC. 
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Parts Search Module 

The parts search module provides the user with information that is read from 
a parts database, whether local to the computer or fixed storage medium or available 
through the communications module. The parts database can be in Microsoft Access 
5 or Oracle. Search request information is input from the user. A database search will 
occur and records are read and displayed. The matching records are displayed, for 
example, up to 20 at a time. Error messages are displayed if a database error occurs, 
the data is not in the expected format, or if no data is entered. If no data is found a 
message is displayed to that effect. 

10 The parts locator or data module is now described in coimection with Figure 

7. The display parts menu 700 may be a custom foraiat for each task that a user needs 
to perform. The program determines the database format, e.g., Microsoft Access or 
Oracle. The program will read the header information to determine the fields and 
field lengths. The search will not be performed if the program caimot open the 

15 database, the database is not readable or is not in the correct format. 

The parts program may be called display parts 700 and is accessed through a 
display entry screen 700 in which the user is prompted to selection 702 from a two 
options: locate parts 704 or search parts 706. If the user selects the locate parts 704 
option, then the user may select the contents from a menu in a browser window 708 

20 that provides the user with options 710. The options 710 available to the user may 
include hyperlink controls 712 or customizable browser controls 716. 

If hyperlink 712 is chosen, then a context sensitive display 714 is'activated 
that provides the user with specific information about the parts available. For 
example, MPEG video, AutoCAD drawings, viewer and schematics, etc., are 

25 displayed for the user. Alternatively, the user may select to enter a browser control 
716 window though which the user select may elect to operate the functions of the 
browser, such as home, refresh, back, print, fmd, etc., that are enabled through the 
HTML menu page 718 

Alternatively, the user is prompted to search the parts database 706 or submits 

30 user provided data to be searched at a display entry screen 720. The user may then 
make a selection from the criteria provided by the user or made available to the user 
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at retrieve search criteria 722, The user may then select from three options at 724. 
If the user decides to submit the criteria input or selected by the user 726, the criteria 
is directed to a query database 728 for an exact, partial or related search parameters. 
If the user makes an error in input the user may cancel the search at 740 which leads 
5 the xiser to the main menu at 742. Alternatively, if the user makes an error the fields 
may be reset and a set search blank criteria is presented to the user at 746. Reset 
clears any and all search fields 722 input by the user, thereby resetting the search 
parameters. If no data is entered an error is displayed, which then prompts the user 
to once again input user data. 

10 If the user provides input data, such as part name, number, catalog number 

and the like, the parts program searches the database for the corresponding part. The 
search results are then displayed 730. If no records match a message is displayed that 
no records were foxmd using the user input search parameters and the user is provided 
with an options menu 732, &om which the user may select a new search 734, the next 

1 5 20 search hits 736 or to end the search 738. The next 20 hits are provided at 742; if 
20 or a fractions thereof remains. If the user selects to end the search, then the user 
is returned to the main screen 742, wherein the user is allowed to choose from the 
main menu options. 

The user may input a data request directly or select fix)m a "call-out" number 

20 that is linked to an image of the machine, process or operation that a user is 
performing and that the user accessed through the SIMON engine. The program will 
get the data entered and send a message that contains the user input to parts_search 
to start the database search. If the search yields a match or several matches then the 
program displays a screen to the user with the fu^t 20 data search output matches. 

25 If the user asks for more data a message is sent to parts_search. Parts_search 

will send more data and it is displayed. If the user is finished, a message is sent to 
parts_search and the program ends. This option allows the user to look at however 
many screens are part of the parts search information for that particular part. 

A number of fimctions may be provided that form part of the parts search 

30 module. 
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begin_search - opens the database, call search_records. When finished is 
received from display_parts, the database is closed. If the open is not successful, an 
error message is sent to display j)arts. 

search_records - finds the first 20 records that match the data received and 
5 send them to display_parts. If no records are found a message is sent to display_parts 
indicating no records were found. While the user wants more records the next 20 are 
sent to dispiayjarts. If the read is unsuccessfiil, an error message is sent to 
display_parts. 

display_screen - displays the screen for user input; 
10 user_response - receives user input, formats data and sends to parts search. 

display_data - receives data from parts search and displays the results. This 
fimction displays messages to the user 

When the parts search module receives data from display_parts, it will open 
the database, read data and look for a match. It will send, e.g., the first ten items 
1 5 found to display j)arts and wait for a message from display_j)arts. If the user asks for 
the next records, the program will look for the next 20 records and send them to 
display_parts if found. If records are not foxind a message is sent to display_parts. 
If the user is finished, the program will close the database. 

The database format for use with the parts search module may be task 
20 specific. As with all the other modules, the user input may be voice, keyboard or 
mouse. Information is read from a parts database that may be in Microsoft Access 
or Oracle formats. Information is input from the user directly or by selecting "call- 
out" numbers associated with an image- 
In operation, after choosing search, the user will enter data to search for in the 
25 database. The user may submit the data, reset the entry fields or cancel the search. 
The entry screen will appear. A database search will occur and records are read and 
displayed. The search will look at all fields in the database to make a match. If a 
match occurs, the first 20 records are displayed with menu options that allows the 
user to look at the next 20 records, perform a new search, or end the search. If the 
30 next option is selected, 20 more records are displayed. The user is able to select a 
previous option to look at the previous 20 records. If there are no previous or next 
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records, the previous or next option will not be selectable. The search results screen 
will appear as follows. 

Finally, again with reference to Figure 7, if a database error occurs an error 
message is displayed. This message is displayed if the program is unable to open or 
5 read the database. If the data is not in the expected format, it is unreadable data. If 
the user has the ability to connect to the customer support center (CSC), there is an 
option to connect to the CSC to obtain the access to the local database or even access 
a database on-line. If no database is accessible an error message is displayed 
prompting the user to contact the CSC. 

1 0 The remote on-line data are stored in either Microsoft Access or Oracle. The 

program determines the database format, Microsoft Access or Oracle and 
communicates it to the user over, e.g., a wireless network. The program will read the 
header information to determine the fields and field lengths. In case of an error 
message, the display error message is displayed if the data from the database is not 

15 accessible or beyond the user's access level. The user is able to connect to a CSC to 
obtain access or the information directly. 

A graphic part locator is selectable from the parts menu. The content may be 
customer, machine or operation specific. The graphic file format may be, e.g., .dwf 
(drawing web files), ,bmp (bit map), .pcx (PC Paintbrush), .gif (graphics interchange 

20 format) or .jpeg (joint photographic experts group). The source data for a particular 
machine may be provided by the customer or may be developed apart or in 
conjunction with the end user or machine manufacturer. The parts locator module 
212 may be written in HTML. The Whip plug-in is used to view files in .dwf format. 
The parts locator module serves to display machine parts or operational steps 

25 with "call-out" numbers that allow the user to look at the next level of parts break 
down. For example, once one part containing a number of sub-parts is identified then 
the user may call out the nimiber or nimibers of the sub-parts to observe, e.g., their 
interaction within the larger unit of parts. This parts breakdown display provides the 
user with blow-ups and assembly information for the part(s) as well as "call-out" 

30 numbers for the parts. The part information from the database may be displayed, e.g., 
at the bottom of the display for ordering purposes. The part number may be looked 
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up directly and ordered or located. Alternatively, the user may chose to back-up and 
display the previous part break down at a higher level. 

The HTML page displays graphics of the machine and parts. The user is able 
to view breakdowns of the parts by choosing call-out numbers defined on the graphic 
5 and may even view their interaction or assembly video using instructional MPEG 
video or other images. 

The HTML page will call a Visual Basic program, passing it the part 
information to locate in the parts database. The part data is located in the database 
and displayed on the screen. The user may input by voice, keyboard or mouse. The 
10 user v^U select Parts from the button menu. The parts sub-buttons will appear as 
follows. 

The voice input may be the user choices, such as, call-out numbers. The call- 
out numbers are displayed at the next level for a parts break down. Back displays the 
previous part break down. Help is available from the parts locator module. 
1 5 Alternatively, the user may chose to exit the parts locator module and close the part 
breakdown information. 

The image graphics file can be dwf, bmp, pcx gif or jpeg. The data is stored 
in either Microsoft Access or Oracle. The program determines the database format, 
Microsoft Access or Oracle. The program will read the header information to 
20 determine the fields and field lengths. The matching record is displayed. 

A message is displayed in the bottom frame if the data from the database is 
not accessible. The parts locator displays a graphical parts locator of the machine and 
part information. The parts locator module program may be written in HTML and 
may contain a top down break down section that has one or more security levels set 
25 up in conjunction wdth or independent of the security module fimctions. 

Reports Module 

The primary duty of the reports module is to generate hands-ofif report logs 
that reflects the task-specific preventive and repair maintenance of SIMON users. 
30 The module also captures a database log of visited screen titles. Report data may be 
gathered on the initial activation of SIMON and stored in, e.g., Microsoft Access 
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database format. The date and time of the report capture will equal the system's date 
and time at log-on and indicated accordingly on the report. Failed performances are 
also captured and recorded. The report module may operate independent of the user 
in addition to being transparent to the user. Else, the user may activate and inactivate 
5 the reporting function depending on their access level. 

Users have the option to generate a daily or custom report from the Report 
button sub-buttons. The daily reporting option prints a daily log of all activities 
related to preventive and repair maintenance specific to that day. The custom 
reporting option in the report submenu allows the user to enter the desired time period 

10 to print the report. This means that the user is required to specify the beginning and 
ending periods needed for "printing" under the custom option. The Custom option 
also allows a filtering option that enables a selection of preventive maintenance only, 
repair maintenance only, or both. Printing includes both hard copy and electronic 
data reporting functions. 

1 5 The accuracy of transmitted data and requirements data are based on report 

data generated by the system for, e.g., the Microsoft Access database and may be 
task-specific. Validity of the data is determined firom the recording of information 
within the system report log. 

Response time for screen title log capture is less than one second. Response 

20 time for printing the Report module will vary depending on the type of printer models 
used or communications link provided. Response time is also dependent on the 
frequency of SIMON activity that is captured in report logs, as well as the length of 
the report or if a number of reports are being submitted at one time. 

Figure 8 is a flow chart of the basic reporting functions of the SIMON engine. 

25 A Reports button is available fi*om the button menu at the main SIMON engine 
screen (as are the other modules disclosed herein) and provides access to the reports 
menu options 900. Selection of the sub-button options imder Reports via voice or 
mouse method initiates the opening of specific or generic electronic reports. There 
are two selections firom which to choose - Daily 902 and Custom 904. Selection of 

30 the Daily option 902 prints a report of, e.g., the daily log record. The daily log report 
may be a prompt for the user to conduct that maintenance or repairs on that day, or 
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may serve to report the users maintenance and repair of the day that passed. Selection 
of the Custom option prints a report specific to the user's begmning and ending date. 
The printed report, or an electronic version of the report, may be submitted vwth the 
communications module for viewing by the user, to network, to one or more serves, 
5 to the customer support center or to any other software package or user that is in need 
of the report. For example, the finished report may be e-mailed to a remote location 
for gathering, analyzing, storing and acting upon the reported data. 

Selecting to print 906 a particular report invokes the above selection box. A 
printer must be attached if a hard copy printout is desired. If only a single printer is 
1 0 available, that printer is displayed in the printer text box. If there are multiple printers 
available, the user will have a box &om which to choose the desired printer. The user 
may wish to print out another report or a copy of the present report at 908. The 
following are the detailed functional and performance requirements for a reports 
module. 

15 DAILY — allows the user to print the production day's report log. This 

fimction is strictly limited to the current day's log activities. Requirements other than 
the daily activity should be referred to the submenu function "Custom." Selection of 
"Daily" option will default v^dth the current date. 

CUSTOM - allows the user to directly specify the exact date of desired 

20 activity. Users may enter a beginning time period and an ending time period to print 
the log report. For a printing of the current day's log, the "Daily" option is used. 

HTML Page: allows the user to print-out the active HTML page as displayed 
on the SIMON screen. 

The data obtained firom the user during a specific fast and maintenance or 

25 repair of, the machine, process or operation that is needed may be provided via inputs 
generated based on log capture activity or via user input. This is directly proportional 
to the user's firequency of usage. Outputs may be generated with hardcopy reports. 
In case of failure, the reports menu or failed access to the printing of the report items 
may be solved locally or via contact with a customer support center. 

30 If the custom option is selected at 904, then the user enters the beginning and 

ending period for the custom report 910. Alternatively, the user may be prompted to 
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request different forms or types of reporting, e.g., a more condensed report for 
transmission via wireless LAN. The report log print-outs are customized according 
to the customer's desired output 912. Modifications are to be managed by the 
customer for any subsequent changes to the report format. 
5 Because of the importance of maintaining, filing, keeping and in some cases 

^maintaining security, a mandatory security login 914 requirement may be necessary 
for access to print log reports whether from daily reports or from the custom reports 
sub-routine of the reports module 214. Without login authority, the reports submenu 
option is imavailable and disabled. 

10 

Security Module 

The component module of the SIMON engine that maintains secxirity to and 
from the program is the security module 216. The security module is described in 
conjimction with Figure 9, which is a flow chart of the security protocols. The 

1 5 primary duties of the security module are to safeguard software integrity and provide 
maintenance accountability. The mode of security performance is defined as follows. 
First, as the SIMON engine is started 1000 the user is prompted to log-in 1002 using 
their user identification nximber and if required a password. The process may involve 
two steps, first the user number is requested and then the user password 1004. If the 

20 login and password are correct then the user is allowed to access the SIMON system 
1006. 

When the security module is a mandatory requirement it allows users to enter 
a login and personal password to verify and authenticate access. All users must gain 
access through a valid usemame and password. An allowance of three attempts, e.g., 
25 may be provided. 

Part of the SIMON system permits the user to enter the administration option 
1006 within, e.g., the file menu or a separate menu. The administration option 1006 
requires that the user provide an administration login 1008, having the same or more 
stringent security requirements than the login user identification 1002 and password 
30 1004 necessary to access the main SIMON engine program. 
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If a user enters an incorrect login and/or administration password on the initial 
attempt, the program may allow the user two additional attempts before returning the 
user to the SIMON engine. If a correct login and password is verified and 
authenticated, system access into SIMON administration is granted. If the user 
5 wishes to make changes 1016, and/or updates 1018 and/or delete 1020 users and 
passwords for authorized users as well as activate/deactivate the Machine type input 
box 1012 or delete a user/log-file 1018, the user must provide a valid ID-password. 
A valid ID-password allows the user access to the Administration Edit screen 1010. 
Users are able to access options of SIMON'S interface menu system. Access into the 

1 0 software's configuration, however, is disallowed. Only developers with a developer's 
password are allowed to configure system functions. Developers are able to 
customize configuration features accordingly, for example, an administrator may 
reach the set administration 1014 screen in which administration access may be 
completely forbidden, especially from users. 

15 For high level security requirements encryption of passwords and login 

numbers may be used. Higher security may be achieved using single-key usage 
coding and encryption. Accuracy of transmitted data and requirements data are based 
on data for a specific task or provided by a user. Validity of data may be determined 
by the comparison between given data and the software's embedded data. The 

20 following are specific examples of commands and their functionality as it relates to 
the security module. 

<USER> — Login Usage_ Login with a password is a mandatory 
requirement to access SIMON's software system. Access is restricted to non- 
authorized users. Attempted log-ons are Hmited to three tries before system lockout 

25 occurs. A manager's level access is required to restore system functionality in the 
event of system lockout. Levels of authorized access are assigned according to user's 
access needs. There are limitations of user's navigation area within the software (i.e. 
no access in the configuration environment). Based on predetermined design 
specification, the user is restricted to modules defined by the developer. 
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<DEVELOPER> ~ A developer's password enables the developer to access 
SIMON'S configuration to add/update/change/delete any aspect of the coded modxiles, 
thereby allowing the developer to customize SIMON's configurations accordingly. 

LOGIN — allow users to enter a personal identifier for individual authorized 
5 access. Log-ins require alphanumeric entries with a minimxmi of, e.g., five and a 
maximum of ten characters. Alphanumeric entries may or may not be case sensitive. 

PASSWORD — allows user to enter a personal code for individual 
authorized access. Passwords require alphanumeric entries with a minimum of, e.g., 
five and a maximum of ten characters. 
10 PASSWORD VERIFICATION - allows user to update or change personal 

code. The system administrator determines the fi-equency of password changes. 

The specific security features for use with a particular user of the SIMON 
engine may be configured to customer's specifications. Only inputs of login and 
password data are entered using, e.g., voice or keyboard input. Also, login and 
15 password data may be stored for purposes of authorization and authentication. In 
case of a system or software failure, access to the SIMON system will require a 
manager's level access or above to restore. Other failures will require coordination 
with the Customer Support Center for support. Dependent upon the level of desired 
security, encryption modes between 128 and 256 bit is used to insure stable system 
20 integrity. 

For high level security tasks, it may be mandatory that system developers and 
users assign the initial assignment of level logins and passwords. Administrators may 
change this using high security access codes. Security level access may be partitioned 
into, e.g., four domains: 
25 1 Administrator domain - highest authorization access; has 

access rights in all domains; no restrictions are placed on the 

Administrator security level. 

2 Manager domain - has access rights to all areas except the 
ability to assign security level access for users. 
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3 Supervisor domain - has access rights to most areas; the 
Manager and/or Administrator will define the unauthorized areas of 
access. 

4 User domain - has limited authorized access; the Supervisor 
5 and/or Manager will define areas of access. 



Referring once more to Figure 9. If a user enters an incorrect login and/or 
administration password on the initial attempt, the program will allow the user two 
additional attempts before disabling system access 1022. If a correct login and 
10 password is verified and authenticated, system access into SIMON is granted. Valid 
ID-password will allow the user access to the Administration Edit screen 1 024. Users 
are able to access options of SIMON' s interface menu system. Access mto the 
software's configuration, however, is disallowed. 



1 5 Communication Module 

Local . SIMON may be fiilly voice-activated, that is, functioning primarily on 
voice commands. A manual keyboard or even visual tracking interface is optional as 
the input and output device used by the SIMON user to communicate and provide 
commands to the computer. Likewise, the same input/output capability may be used 

20 to access the communications capabilities of the SIMON engine with local users and 
a local network. Voice operation enables users to navigate SIMON' s GUI interface 
hands-firee. Reserved words may be saved for specific SIMON commands, e.g., open, 
close, print, send and exit. Input/output devices for the construction of a "portable 
computer", that is a computer sufficiently lightweight and designed to be worn by a 

25 user, and may be, e.g., the MENTIS computer (Interactive Solutions, Inc.), weighing 
about 2 to 5 pounds (exclusive of the battery pack) and is five and a half by seven and 
a half inches in size. The computer may have a 266 MHz pentium processor with 1 28 
megabytes of RAM and a foiu", six or eight gigabyte hard drive. The computer may 
also be connected to a portable CR-ROM, ZIP, DVD or like large scale storage 

30 media. Through a wireless connection, the large scale storage medium may even be 
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at a remote location. Other hardware that may be interfaced with the portable 
computer via SIMON includes: 

1) VideoCam: Sony CVX-Vl 

2) on head set: VGA screen with built-in earpiece 
5 3) hand held: 6.4 by 1 0.4 inch flat panel display 

4) external audio speakers 

5) on headphone/microphone set: LabTec Clearvoice™ 

6) LAN hardware: WaveLAN®, VBX/CODEC 

Accuracy of transmitted data and requirements data are based on equipment 

10 installed per each component of the communications hardware. The accuracy of 
transmitted data may be checked using systems and software known to those of skill 
in the art of data verification. 

The voice recognition software used by the communication module may be 
the AT&T Watson Voice Control communications software designated for SIMON. 

15 The AT&T Watson Voice Control software application may be embedded within 
SIMON allowing users to speak commands through a microphone via headphones or 
one attached to SIMON's unit. Alternatively, Dragon Software's 
NaturallySpeaking® may be used and has the advantage of accepting natural 
language commands, which allow users to edit and format documents by speaking 

20 commands in a more natural way. It also has an enlarged active vocabulary, and 
Dragon NaturallyMobile™ software, which makes it easy to create documents using 
a hand-held microphone that can also be used to record any communications. 

The Dragon NaturallySpeaking® software application, for example, enables 
users to navigate SIMON's menu and sub-menu options with reserved spoken words. 

25 Non-keyboard interaction may be required which will permit the user to engage in 
other tasks as well as offering mobility within the work environment. The types and 
specifics of a user's voice may be stored and accessed from reserved vocabulary 
commands that are pre-input into the communications module. Voice recognition 
may even be implemented as a digital signal processor. 

30 Remote . Any communication to a CSC is accessed via SIMON. Likewise, 

peer to peer communications may be accessed through SIMON to provide 
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communication over a wireless LAN/WAN between SIMON users. One such 
connection may be through a WaveLAN® card using the NetMeeting software. 

The following is a list of ftinctionalities and equipment for use with the 
present invention. 

5 VideoCam This unit allows parties other than the user to visually see the 

same screen information as the user, for example, a Sony CVX-ll. 

Audio - This unit will allow the user to hear speech and/or sound and 
interact with the SIMON system via voice activation. 

LAN -- This connection will allow the SIMON user to connect remotely to 
10 a LAN for network interface. 

Modem — This unit enables the user to auto dial an IP address to the CSC 
or other phone lists. 

In the event that SIMON'S voice recognition functionality is disabled, the unit 
may be automatically connected with CSC to allow troubleshooting. If further 
1 5 connection fails a manual call in to a CSC may be required to diagnose the problem. 
If it is a minor hardware failure, a camera connection with a CSC representative may 
be used to diagnose the problem via video. Video or audio access will afford a 
solution (i.e., replacement of part) on the failed hardware component. If it is a 
software failure, remote connection via LAN will allow a CSC representative to 
20 upload software directly or allow the user access to that functionality. As a last resort 
the user may have to re-install the software. 

While it may be readily apparent, it is not always easy to interface individual 
hardware components to be compatible with individual communications components. 
This is a problem especially when using equipment from different vendors. The 
25 present inventors have overcome that problem by using a central program to interface 
all the hardware components. 

In operation, the communication module is described in connection with 
Figure 10, in particular with a connection with, e.g., a network, server, or other 
SIMON users a CSC. Once a connection with a CSC 1 1 00 is established the user can 
30 determine whether or not they want to include video imaging capability over the 
wireless LAN, WAN or internet. If a visual connection is required, e.g., when the 
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operator at the CSC must also be able to see the operation of the machine and the 
operators predicament, then the video camera is connected 1102, else the video 
camera is disconnected 1 104. Likewise, the connections to a LAN can be made 1 106 
or disconnected 11 08, a modem connected 1 1 10 or disconnected 1112 and audio may 
5 be connected 1 1 14 or disconnected 1116. Commimications with a CSC provide the 
user of SIMON and the SIMON engine to access in real-time a broader and often 
more experienced workforce that can help the user repair a particular machine. 

SIMON is particularly useful when a user at a remote location is presented 
vAih a task when dealing with wires or equipment that is color coded. The use of a 
1 0 color video camera with sufficient resolution and transmission rate will be required 
to provide the CSC with valuable, and perhaps critical information. The use of a 
color camera will be particularly important to reduce the risk of injury to maintenance 
and repair persoimel that may not be aware of the operational dangers of a particular 
piece of equipment or process when commimicating with a CSC in real time. 

15 

Diagnose Module 

The diagnosis module provides the SIMON engine specific information and 
routines, protocols or checklists that aid in machine diagnostics. The diagnostic 
module of the SIMON engine is described herein in connection with Figure 1 1 . The 

20 machine diagnostics process begins with an error code or codes being produced by 
the machine, operation or process that is in need or repair or maintenance at 1200. 
This error may be provided for in software in the resident machine and input directly 
into Simon by the user. Alternatively, the error code entry screen 1202 is chosen 
from which the user may view and retrieve error codes 1204 fi^om a list or lists or 

25 codes. The list may be arranged in a number of configurations, such as alphabetical, 
numerical, by machine operation or part grouping, etc. To evaluate and determine 
what problems may be occurring on the machine the user then accesses a database to 
get to the matching, appropriate or related HTML links 1206 that are specific or 
related to the error codes. Next, a dynamic HTML menu page is written to disk 1208 

30 and the user reaches a browser window 1210 that provides the user with options 
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1212, which permit the user to access, for example, hyperlink controls 1214 may also 
have standard and customized browser controls for further navigation 1218. 

If hyperlink 1214 is chosen, then a context sensitive display 1216 is activated 
that provides the user with specific information about the content of help available. 
For example, MPEG video, AutoCAD drawings, viewer and schematics, etc. are 
displayed for the user. Alternatively, the user may select to enter a browser control 
1218 window though which the user select may elect to operate the functions of the 
browser, such as home, refresh, back, print, find, etc. that are enabled througji the 
HTML menu page 1220. 

Based on an analysis of the operational situation of a particiilar user and 
machine, process or operation is determined. The diagnosis module will have access 
to machine, process or operation specific information that is part of the task specific 
database that the SIMON engine is able to access. 

The diagnostics module may merely function as a troubleshooting guide, but 
in particular cases and instances may be designed to be a fully integrated, multi- 
functional, multi-media system that guides the user through the diagnostic procedures 
of a particular piece of equipment, operation or process. 

For example, a menu of common problems encoimtered with a particular 
operation may be displayed as a selectable menu. If no common problems are found, 
a common problem list (which may be searchable) is displayed and the user is 
prompted to select an option. Alternatively, the user may access a help menu 1210 
either firom the common problems list or fi-om a more generic problems list. If a 
particular problem is selected by the xiser, then the maintenance procedure for that 
equipment, machine, operation or precess is displayed. Different levels of 
explanation may be presented, depending on the complexity of the task or the user's 
need for instructional help. 

The user will be provided with a task specific diagnostics file and file format, 
e.g., the common problem list. The diagnostics module may be written in Microsoft 
C+-h version 6.0. The diagnostics module will generally be task specific. For 
example, the customer may also provide the machine diagnostics file and information 
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on interpreting the file. The customer provides a common problem list, vMch might 
include non-diagnosable problems. 

The diagnostic module will read the reported error data provided by the 
system or the user and determine what problems are being reported or observed on 
5 the machine, process or operation. A problem menu is displayed with the results. If 
no problems are reported, a common problem list is displayed from which the user 
may select maintenance or repair information. 

One example of a diagnostics program is called KBS (Knowledge Based 
System). When the diagnostics program is called, the machine diagnostics file is read 
10 and analyzed. The KBS will read a table to match the analyzed data to a problem 
description and HTML (HyperText Markup Language) file name. As with all the 
other modules disclosed herein, the diagnostics module may be implemented in 
software or hardware. 

The following are a number of specific commands that may be used with the 
1 5 diagnostic module. 

Diagnose — opens and reads the machine diagnostics file. The data is 
analyzed, depending on the customers or task file specifications. Problem numbers 
are sent to get_problems. If no problem numbers are foimd, a common problem list 
is displayed. 

20 Get_problems — opens a kbs.tbl file and matches the problem nimibers 

received to a problem description and a html file name. This information is sent to 
the calling html page so that it can display the problems and associate an html page 
to each problem number. 

25 Learning Module 

The learning module may be customer, machine, operation, equipment, or 
process specific. The learning module may be developed using, e.g., Macromedia 
Authorware. While the customer may provide the source data for the learning 
module, custom learning systems may be developed to service particular system 
30 needs. 
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An interactive learning application may be developed, in Macromedia 
Authorware for customers choosing this module. The learning module may be an 
interactive training application and can include animation, sound, hot spots, 3D 
graphics, simulations, quizzes, quiz grading, video clips, show me, interactive 
assembly / disassembly of components and tracking student progress. 

The learning modxile is an interactive learning tool that serves to teach, in real- 
time a SIMON user the detailed operation of a machine or process and the manner or 
carrying out repair or maintenance operations without the need to resort to hard 
copies or user's manuals. Training screens are displayed and allow users to interact 
with the application to learn about the customers system. 

While this invention has been described in reference to illustrative 
embodiments, this description is not intended to be construed in a limiting sense. 
Various modifications and combinations of the illustrative embodiments, as well as 
other embodiments of the invention, will be apparent to persons skilled in the art 
upon reference to the description. It is therefore intended that the appended claims 
encompass any such modifications or embodiments. 
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What is claimed is: 

1 . A portable instruction system for use by a user, comprising: 

a computer sufficiently lightweight and designed to be worn by a user; 

a memory connected to and imder control of the computer; 

a display device that can receive display signals from the computer for visual 
display to the user; 

an input device by which the user enters commands to the computer; and 

an instructional program that the computer accesses and stores in memory in 
response to a user command and displays information concerning a task to be 
performed by the user on the display device in response to commands from the user. 

2. The system of claim 1 , further comprising a garment suitable for securing the 
computer on the user's body and means for mounting the computer on the 
garment. 

3 . The system of claim 1 , wherein the instructional program comprises: 

a program engine that contains a program logic, which program logic being 
loading into memory, whereby the program engine includes one or more user 
assistance programs that may be accessed by the user through the program engine and 
the program engine can access a database containing data that is specific to the user's 
task. 

4. The system of claim 3, wherein the instructional program comprises one or 
more search modules each accessible by the program engine for searching a 
database containing data that is specific to the user's task. 

5. The system of claim 1, wherein the task comprises a manual operation 
undertaken on an apparatus. 
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6. The system of claim 7, wherein the manual operation comprises repair or 
maintenance of the apparatus. 

7. The system of claim 1 , wherein the task is performed at a location where the 
apparatus is installed for use. 

8. The system of claim 1, wherein the task is performed at a location on the 
apparatus at which a standard desktop computer cannot be accessed by the 
user at the same time as the task is performed. 

9. The system of claim 1, wherein the task comprises a manual operation 
undertaken on an apparatus, and a task module is loaded in memory that 
includes data specific to the apparatus. 

1 0. The system of claim 1 , wherein the instructional program includes program 
logic permitting the user to select an image of one or more parts of the 
apparatus to view on the display device, 

1 1. The system of claim 10, wherein the instructional program includes name 
data identifying one or more parts of the apparatus that the user has selected 
to view. 



1 2. The system of claim 1 1, wherein the data includes instructions for assembly 
or disassembly of the one or more parts of the apparatus that the user has 
selected to view. 



13. The system of claim 1 1 , wherein the data includes instructions for repair of 
the parts of the apparatus that the user has selected to view. 

14. The system of claim 11, wherein the data includes instructions for fault 
diagnosis of the parts of the apparatus that the user has selected to view. 
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15. The system of claim 11, wherein the display device includes means for 
temporarily mounting the display device in proximity to the parts the user has 
selected to view. 

16. The system of claim 1, further comprising a data retrieval device connected 
to and under the control of the computer, which data retrieval device accesses 
a data storage medium on which all or part of the instructional program is 
stored. 

1 7. The system of claim 1 , further comprising: 

a data entry system connected to the computer by which the user can enter 
data into the computer; and 

a data storage and retrieval device connected to and under the control of the 
computer on which data generated by the user is stored; 

wherein the data entry system is connected to the computer and the user can 
enter data into the computer which is then stored on the storage medium. 

1 8. The system of claim 1 , wherein the instructional program that the computer 
accesses and stores in memory is uploaded to the computer via a wireless 
communication. 

1 9. A portable instruction system for use by a user, comprising: 
a computer of a size sufficiently small to be worn by a user; 
a memory connected to and imder control of the computer; 

a display device that can receive display signals from the computer for visual 
display to the user; 

an input device by which the user enters commands to the computer; 

an instructional program that the computer accesses and stores in memory that 
contains instructional information concerning a task to be performed by the user; 

a video camera of a size sufficiently small to be worn by a user; 

a system which transmits images from the video camera to a remote site; and 
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a commxinications system whereby the user can communicate with an 
assistant that views the images at the remote site concerning the task being 
performed. 

20. The system of claim 1 9, wherein the assistant views the images on a remote 
computer 

21. The system of claim 19, wherein the commxinications system mcludes an 
audio communication device. 

22. The system of claim 1 9, wherein the communications system includes a video 
commimication deyice. 

23. The system of claim 20, wherein the communications system includes a 
receiver for receiving data from the remote computer used by the assistant to 
the user's computer; and 

a transmitter for transmission of data from the user's computer to a remote 
computer used by the assistant. 

24. The system of claim 19, wherein the computer accesses and stores the 
instructional program in memory in response to a user command, and 
information concerning the task to be perfonned by the user is displayed on 
the display device in response to commands from the user. 

25 . A computer commimication system, comprising : 

a computer of a size sufficiently small to be worn by a user; 
a memory connected to and under control of the computer; 
a display device that can receive display signals from the computer for 
5 visual display to the user; 

an input device by which the user enters commands to the computer; 
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a wireless data transmitting and receiving station having a range, which 
station can transmit data to and receive data from the computer anywhere within 
the range; 

a wireless data transmitting and receiving device connected to the 
computer, which device can transmit data to the station and receive data from the 
station; and 

a computer network that can send data to and receive data from the 
wireless station while the user is working on the task within range of the wireless 
station. 

26. The system of claim 25, further comprising a garment suitable for securing 
the computer on the user's body and means for mounting the computer on 
the garment. 

27. The system of claim 25, wherein the instructional program comprises: 
a program engine that contains a program logic, which program logic 

being loading into memory, whereby the program engine includes one or more 
user assistance programs that may be accessed by the user through the program 
engine and the program engine can access a database containing data that is 
specific to the user's task. 

28. The system of claim 27, wherein the instructional program comprises one 
or more search modules each accessible by the program engine for 
searching a database containing data that is specific to the user's task. 

29. The system of claim 25, wherein the task comprises a manual operation 
undertaken on an apparatus. 

30. The system of claim 29, wherein the manual operation comprises repair or 
maintenance of the apparatus. 
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3 1 . The system of claim 25, wherein the task is performed at a location ^^^e^e 
the apparatus is installed for use. 

32. The system of claim 29, wherein the task comprises a manual operation 
undertaken on an apparatus, and the task module includes data that 
comprises images of the apparatus. 

33. The system of claim 25, wherein the wireless data transmitting and 
receiving device transmits and receives communications between a user 
and a customer support center at a remote location. 

34. The system of claim 25, wherein the wireless data transmitting and 
receiving device transmits and receives data through a VBX/CODEC box. 

3 5 . The system of claim 25, wherein the wireless data transmitting and 

receiving device transmits and receives data through a hardwired network 
that connects via TCP/IP through a secure intemet link to the remote 
location. 



36. 



The system of claim 25, wherein the wireless data transmitting and 
receiving device is a cellular phone. 
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37. A computer communication system, comprising: 

a first computer of a size sufficiently small to be worn by a user; 

a memory connected to and under control of the first computer; 

a display device that can receive display signals firom the first computer for 
visual display to the user; 

an input device by which the user enters commands to the first computer; 

a wireless data transmitting and receiving device coimected to the first 
computer, which device can transmit data; 

a wireless data transmitting station, which station capable of receiving and 
transmitting data fix)m the station; 

a second computer of a size sufBciently small to be worn by a user; 

a memory connected to and under control of the second computer; 

a display device that can receive display signals fi'om the second computer 
for visual display to the user; 

an input device by which the user enters commands to the second 
computer; and 

a wireless data transmitting and receiving device connected to the second 
computer, which device can transmit data to, and receive data fi:om, the wireless 
transmitting station, whereby the data is generated by the user wearing the first 
computer. 
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38. An instructional computer data signal in a carrier wave, comprising: 

a source code segment comprising an instructional program that controls a 
computer sufficiently lightweight and designed to be worn by a user, which source 
code segment being stored in memory, containing task specific instructional 
S information concerning a task to be performed by a user at a location and 

controlling the interaction between the computer and a display device and at least 
one audio input and output device imder the control of the user; 

a wireless data transmitting and receiving a carrier wave through a device 
connected to the computer, which device can transmit data to the station and 
1 0 receive data from the station; 

a wireless data transmitting and receiving the carrier wave through a 
station having a range, which station can transmit data to and receive the carrier 
wave from the first computer anywhere within the range; and 

a computer network that can send data to and receive data from the 
1 5 wireless station while the user is working on the task within range of the wireless 
station. 

39. An instruction system, comprising: 

an instructional program that controls a computer and is stored in memory, 
which program comprises one or more program modules which contain 
instructional information concerning a task to be performed by a user at a location; 
5 an input/output program that controls the interaction between the computer 

and a display device and at least one audio input and output device; and 

a communications interface that interconnects the user, the computer and 
the one or more program modules, whereby the user is able to interact through the 
commimications interface in real-time with each of the one or more program 
1 0 modules through the input/output program. 

40. The system of claim 39, wherein the input/output program displays an 
image of a component in need of user intervention and that includes an 
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image of a specific component that is specific to the user's task at the 
user's location. 

4 1 . The system of claim 39, wherein the input/output program displays 
specific parts information regarding a component of the user's task at the 
user's location. 

42. The system of claim 39, further comprising a wireless communications 
program that provides for data interpretation and transfer between the 
computer and a remote computer through a computer modem. 

43. The system of claim 39, wherein one module of the instructional program 
is a parts search module that permits the user to access a database of parts 
names and numbers specific for the user's task at the user's location firom a 
storage mediimi. 

44. The system of claim 39, wherein one module of the mstructional program 
is a maintenance procedure module that provides the user access to a 
database of information at a user's location user's task that is specific to 
the user's task for which the user requires instructional assistance. 

45. The system of claim 39, wherein one module of the instructional program 
is a help module that provides the user with information regarding the 
fimctioning and parameters for using the instructional program and one or 
more program modules. 

46. The system of claim 39, wherein one module of the instructional program 
is a reports module that provides the user with electronic reporting forms 
that the user completes using voice commands. 



58 

The system of claim 39, wherein one module of the instructional program 
is a diagnostic module that provides the user with troubleshooting 
information for the user's task at the user's location. 

The system of claim 39, wherein one module of the instructional program 
is a security module that restricts the user's access to the databases and 
functionality associated with the user's task. 

The system of claim 39, wherein one module of the instructional program 
is a learning module that the user can access to obtain step-by-step 
instructional information on the use of the equipment and tools needed to 
perform the user's specific task at the user's location. 

The system of claim 39, wherein one module of the instructional program 
is a main program engine that is able to interface with one or more other 
modules and that allows the user to conmiunicate using voice-recognition 
with the one or more other modules that provide the user information for 
performing a specific task at the user's location. 

The system of claim 39, wherein the computer is connected through a 
wireless modem to a network, the network being capable of audio and 
video data reception with a location that is remote firom the location of the 
user. 

The system of claim 39, wherein the computer is connected through a 
wireless modem to a network, the network being capable of audio and 
video data reception with a location that is remote fit)m the location of the 
user and wherein the remote location is a customer support center that 
provides the user with real-time assistance to the user regarding the task of 
the user at a location. 
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